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Introduction 


Building on the success of the Advanced Placement Program* 
and the findings regarding academic intensity and the qual- 
ity of one’s high school curriculum in preparation for success 
in college (Adelman, 1999), the College Board developed a 
program in the content areas of English and mathematics 
designed to prepare students for challenging content as early as 
grade six and extending through high school. Conceived with 
the goal of creating high-quality professional development 
activities with associated instructional strategies embedded in 
instructional materials rich in content and cognitive demand, 
the resulting Textual Power and Mathematics with Meaning™ 
instructional materials and professional development ulti- 
mately seek to improve student engagement and achievement 
in the classroom. 

Textual Power and Mathematics with Meaning were 
piloted in high schools in academic year 2001-02. In aca- 
demic year 2002-03 Textual Power and Mathematics with 
Meaning were piloted in both middle schools and high 
schools. These pilot programs continued in middle schools 
and high schools in academic year 2003-04. To inform the 
development process, the College Board contracted with 
researchers from the American Institutes for Research (AIR) 
to conduct a formative evaluation of the program. The first 
12-month evaluation phase examined the 2002-03 academic 
year and is referred to as Year 1. The second 12-month eval- 
uation examined the 2003-04 academic year and is referred 
to as Year 2. 

The successful implementation of any educational 
program or policy is dependent on an assumed set of linked 
components that will enable the main actors to effect change 


and desired outcomes (American Institutes for Research, 
2003). The implementation and subsequent effect on student 
achievement is predicated on a set of assumed linkages, 
depicted as follows: 


High Quality 

Professional 

Development 


Instructional 
Materials with 
Rich Content and 
High Cognitive 
Demand 


/ 


Classroom Level 



Implementation 


Improved Student 

of Materials and 


Achievement 

Strategies 




As such, both of these yearlong evaluation studies exam- 
ined the questions of classroom implementation of Textual Power 
and Mathematics with Meaning and student achievement. 


Year 1 Evaluation of 
Textual Power and 
Mathematics with Meaning 

To address the questions of implementation and achievement, 
the Year 1 evaluation relied on these sources of data: 

• Participant Teacher Survey: Administered in three waves 
(Introductory, Reflections, and Final) to a nationally 
representative sample (see Tables 1-3); 

• District Administrator Interviews; 

• Site Visits: Classroom observations in treatment and com- 
parison classes, teacher interviews, principal interviews, 
and student focus groups (see Tables 4 and 5); 
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• Student Work Analyses: Collection and analysis of 
student work from both treatment and comparison 
classes (see Table 6); and 

• Student Achievement Analyses: Using student-level 
achievement data for matched treatment and control 
classes from two districts. 

Implementation Findings 

The Implementation Process 

The teacher is the central implementor of Textual Power and 
Mathematics with Meaning, but districts, schools, and the 
College Board play significant roles in the dissemination and 
support of the programs. Creating an environment for imple- 
mentation begins as the College Board establishes a relation- 
ship with the district, school, and teacher, and supports this 
relationship by producing unique instructional materials and 
high-quality professional development. Drawing from the site 
visit data, findings regarding implementation at the school 
and district follow: 

• Teacher Buy-in. Teacher motivation and enthusiasm are 
central to effective use of the instructional strategies and 
materials, given the degree of autonomy associated with 
implementation. The professional development acted as 
a catalyst for buy-in, receiving praise from teachers and 
administrators; however, participation in decision mak- 
ing regarding the use of the program influenced teacher 
investment and dedication to the use of the materials. 

• Curricular Consistency. Levels of consistency within a 
school or district affect the capability of teachers to effec- 
tively use Textual Power and Mathematics with Meaning. 
While many districts indicated that the program is a 
good “fit” with their goals and standards, other districts 
needed to examine how to incorporate the instructional 
material topics into their curricula. 

• Professional Community. The existence of a trained 
teacher network or interconnected professional community 
is a key factor in maintaining implementation over time. 

• Instructional Leadership. The leadership of either a 
principal or key teacher coordinator was found to be an 
effective method of instructional support. 

• Student Engagement. The perception of student 
engagement together with the belief that Textual Power 
and Mathematics with Meaning add value to instruc- 
tion, strongly influenced teachers to make changes to 
align instructional philosophy and practice with the 
programs. 


Classroom Implementation 

Interview and survey data indicate differences in teachers’ 
patterns of use and perceptions of individual units; how- 
ever, teachers generally perceived the Textual Power and 
Mathematics with Meaning to be of value in content and 
student engagement. 

• Usage Trends. The majority of all teachers reported 
making minor modifications to the units prior to using 
them, with Textual Power users more frequently report- 
ing making modifications. Teachers reported using the 
instructional units with regular students, advanced stu- 
dents, and classes of mixed ability, but few teachers 
reported using the materials with students of the lowest 
ability levels. 

• Usage Differences. Differences between schools, between 
teachers, and between content disciplines emerged. 
Teachers of Textual Power were more likely to treat 
the materials as a comprehensive instructional pack- 
age. Textual Power teachers commented on the need for 
more scaffolding, whereas teachers of Mathematics with 
Meaning felt some assignments were especially challeng- 
ing because of the reading skills required. The importance 
of alignment with standards and curricula emerged. 

• Perception of Value. Despite varying usage patterns and 
alignment concerns, significantly over 80 percent of both 
Textual Power and Mathematics with Meaning teachers 
reported that they felt the instructional materials provide 
a good framework for what students should know and be 
able to do. 

• Perception of Engagement. Despite some questions 
regarding level of challenge of the instructional materials, 
on the Final Survey 86 percent of the Mathematics with 
Meaning teachers and 80 percent of the Textual Power 
users reported that they agreed or strongly agreed that 
their students were actively engaged with the instruc- 
tional units. The interview data reveal that teachers 
attribute this engagement to the group work, the active 
nature of the units, and the hands-on activities. 

Student Work Analyses 

Student work samples from both treatment classes and 
control classes were analyzed. Textual Power teachers and 
Mathematics with Meaning teachers submitted student work 
samples based on the instructional materials and related 
assignments, and the respective control class teachers submit- 
ted assignments they considered typical. The assignments 
were coded based on the degree to which students exhibited 
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different types of conceptual and technical skills using rubrics 
developed specifically for the evaluation of Textual Power and 
Mathematics with Meaning. 

• In all categories coded for the English work samples, 
there was little difference noted between the Textual 
Power classes and the control classes. The researchers 
noted that the analysis was hindered by the fluid and 
creative nature of language and communication, and the 
diversity within the sample of collected work. 

• The Mathematics with Meaning work samples scored 
much higher than the control group work samples in 
three of the four mathematics criteria, conceptual under- 
standing, communication, and problem solving/reasoning. 

Classroom Observations 

The findings from the classroom observations dramatically 
sharpen the findings regarding classroom implementation 
and reveal manifest differences between Textual Power and 
Mathematics with Meaning classes and control classes in sev- 
eral areas. For each separate activity in each classroom obser- 
vation, researchers coded the observational data with a focus 
on materials used, activity organization, teaching strategies, 
and classroom outcomes. Textual Power and Mathematics 
with Meaning classes differ markedly from the comparison 
control classes in the following ways: 

• The number of distinct instructional activities was gen- 
erally greater in Textual Power and Mathematics with 
Meaning classes. 

• Textual Power and Mathematics with Meaning classes 
generally spent more time in interactive classroom 
modes. 

• Textual Power and Mathematics with Meaning teachers 
spent more time guiding student work as opposed to 
lecturing. 

• Student behavior was markedly better in Textual Power 
and Mathematics with Meaning classes. 

• Teachers spent more time on instruction versus class- 
room management or unrelated activities in Textual 
Power and Mathematics with Meaning classes. 

• Student engagement as measured by apparent time- 
on-task was markedly higher in Textual Power and 
Mathematics with Meaning classes. 

• Mathematics with Meaning teachers employed inves- 
tigative learning strategies more frequently than their 
counterparts in control classes and used small student 
learning groups more frequently than the teachers in the 
control classes. 


Achievement Findings 

To examine the relationship of instructional materials and 
instructional strategies associated with Textual Power and 
Mathematics with Meaning with student achievement, 
student achievement data at two points, spring 2002 and 
spring 2003, were analyzed for the same cohort of students. 
A “pretest/posttest with matched control group” research design 
was utilized to determine achievement differentials between 
Textual Power classes and control classes, and Mathematics 
with Meaning classes and control classes. The control classes 
were matched on prior achievement, grade level, and courses. 
Student-level achievement data from two districts — District 
A and District B — were obtained. In addition, analyses were 
conducted separately by school level (middle school and high 
school) and subject (English and mathematics). There were 
seven separate analyses — two school districts by two school 
levels by two subjects, minus District B math in middle school 
where Mathematics with Meaning was not implemented. 
Hierarchical Linear Modeling (HLM) was used for the analyses. 
The results of the analyses found both positive and significant 
effects on student achievement for 2002-03 (Year 1) of the 
evaluation of Textual Power and Mathematics with Meaning at 
the following levels (see Table 7): 

• District A, High School, Textual Power, p < .01 

• District A, High School, Mathematics with Meaning, 
p < .05 

Follow-up analyses of state mean gains in comparison to 
District A gains in both Textual Power and Mathematics with 
Meaning classes revealed that the results in the District A high 
school analyses were not only statistically significant but of an 
important magnitude. These achievement findings provide 
evidence that Textual Power and Mathematics with Meaning 
can effect positive change in student achievement levels. 

Year 2 Evaluation of 
Textual Power and 
Mathematics with Meaning 

The research activities in the Year 2 evaluation are method- 
ologically similar to those undertaken in Year 1. Although 
smaller in scope, the Year 2 evaluation was deeply considerate 
of the Year 1 work, building from those findings but pushing 
beyond the short-term consideration of implementation to 
examine the staying power of Textual Power and Mathematics 
with Meaning. 
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To address the questions of implementation and achieve- 
ment in Year 2, the evaluation relied on these sources of data: 

• Site Visits: Site visits to high schools and middle schools 
included interviews with teachers and administrators as 
well as observations of classes with teachers who were 
using Textual Power and Mathematics with Meaning and 
comparison classes in English and mathematics respec- 
tively (see Table 8). 

• Student Achievement: Student achievement data from 
state assessments — District A and District B that were 
analyzed in Year 1 — were analyzed for Textual Power 
classes and matched control classes, and for Mathematics 
with Meaning and matched control classes. 

Implementation Findings 

From the interview data collected at school sites, several 
themes emerged relevant to the implementation and sustain- 
ability of Textual Power and Mathematics with Meaning. The 
emerging themes follow: 

• Professional Development. From the perspectives of teach- 
ers and administrators, the professional development associ- 
ated with Textual Power and Mathematics with Meaning is 
reported as one of the most exciting and beneficial aspects 
of the program. Teachers from both disciplines describe 
the experience as positive — specifically the useful activities, 
active learning, and the opportunity for collective participa- 
tion. A limited amount of criticism of the content was cited. 

• Pedagogy. Teachers using Textual Power and Mathematics 
with Meaning cite the hands-on approach and collaborative 
nature provided by the instructional activities which provide 
new ideas and creative methods for engaging students in 
nontraditional lessons and texts. Successful implementation 
requires a willingness to try new instructional techniques. 

• Content and Skills. Most teachers using Textual Power 
and Mathematics with Meaning agree that the content of 
the instructional materials are appropriate for their classes 
and place an emphasis on higher- order and critical think- 
ing skills. However, concern regarding basic skills and state 
assessments can force teachers to relegate Textual Power 
and Mathematics with Meaning instructional materials to 
a lower priority to allow for coverage of basic skills. 

• Materials. Textual Power and Mathematics with Meaning 
materials disproportionately were described as com- 
prehensive. The availability of materials emerged as a 
consideration, as well as future funding to continue the 
professional development and implementation of the 
program. 
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School Capacity 

Site visit data revealed the strong influence of school struc- 
tures and conditions on the depth of implementation. The 
findings revealed several school capacity elements to be sig- 
nificant factors: 

• Teachers’ Knowledge, Skills, and Dispositions. Teachers 
across the sites described Textual Power and Mathematics 
with Meaning as grounded in good teaching practices. 
Teachers described favorably the comprehensive nature 
that balances skills with higher-order thinking and prob- 
lem solving. The teachers’ varied descriptions of “useful- 
ness” appears tied to the type of student population with 
which they work. 

• Program Coherence. A significant tension exists 
between a teacher’s desire to use Textual Power or 
Mathematics with Meaning and the perceived necessity 
to address one or more of the following: state standards, 
preparation of students for tests, and mandates for 
other instructional initiatives. Where adoption of Textual 
Power or Mathematics with Meaning is voluntary, (i.e., 
teacher choice rather than district adoption), the ten- 
sion is unavoidable. In schools and districts that endorse 
Textual Power and Mathematics with Meaning initiatives 
(e.g., incorporating into pacing guides), this tension 
appears to be lessened greatly. 

• Professional Community. The presence of a profes- 
sional community facilitates coordinated instruction, 
and teachers find the opportunities for professional 
conversations stimulating, motivating, and productive. 
The availability of the professional community to sup- 
port sustained use of Textual Power or Mathematics 
with Meaning is a function of program coherence and 
instructional priorities (i.e., teachers in the professional 
community recognize the initiatives as closely linked to 
their instructional objectives). 

• Technical Resources. Access to resources is directly linked 
to sustainability. Few sites reported particular difficulty 
with gaining the resources to support the implementation 
of Textual Power or Mathematics with Meaning. This may 
be a function of defining the initiatives as instructional 
initiatives that emphasize instructional strategies and pro- 
fessional development rather than a curricular program 
that requires the resources. Funding for future use remains 
a concern for some districts. 

Classroom Observations 

The findings from the classroom observations describe the 
implementation of Textual Power and Mathematics with 
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Meaning at the level at which the initiatives arguably have 
their greatest impact — in the classroom. To address the 
complexities of classroom dynamics, classroom-based fac- 
tors including teacher- student interactions, types of teaching 
strategies, and student engagement were examined through 
direct and detailed classroom observations. Combining the 
Year 2 observations (see Tables 9 and 10) with the classroom 
observations from Year 1 reveals manifest differences in sev- 
eral areas between Textual Power classes and English control 
classes and Mathematics with Meaning classes and mathemat- 
ics control classes. The differences follow: 

Materials Used: 

• Teachers trained in Mathematics with Meaning used 
more manipulatives than the mathematics comparison 
classes. 

• Teachers trained in Textual Power used more text-based 
material (e.g., fiction, drama, and poetry) than did the 
English comparison classes. 

Classroom Organization: 

• Textual Power and English comparison classes had very 
similar rates of organization in terms of whole-class, 
small group, and individual groupings. 

• Textual Power and English comparison classes had 
similar rates of student focus in terms of passive and 
interactive modes. 

• Mathematics with Meaning classes exhibited higher rates 
of organizational strategies that are more interactive 
(e.g., group work, pair work, and whole-class discus- 
sions) than did mathematics comparison classes. 

• Mathematics with Meaning teachers spent more time 
leading and supporting student work than did the 
comparison classes in which teachers spent more time 
presenting information. 

• Both Textual Power and Mathematics with Meaning 
classes included a greater number of distinct instruc- 
tional activities than the respective comparison classes. 

Instructional Strategies: 

• Textual Power classes used strategies related to Making 
Meaning from Texts and Creating and Presenting Texts 
at a higher rate than the comparison classes which had 
higher rates of Practice and Drill strategies. 

• Mathematics with Meaning classes used Problem Solving, 
Reasoning and Proof, Communication, Connections, 
and Representation strategies at a higher rate than math- 
ematics comparison classes. The Problem Solving and 
Communication strategies appearing most frequently 


were those related to learning, investigating, and practic- 
ing mathematical concepts; discussing or writing math- 
ematical explanations; and clearly discussing or writing 
about mathematics. 

Student Engagement: 

• Student behavior disturbances occurred less frequently 
in both Textual Power and Mathematics with Meaning 
classes than in the respective comparison classes. 

• Both Textual Power and Mathematics with Meaning 
classes had higher rates of student on-task behavior than 
did the comparison classes. 

• Mathematics with Meaning classes exhibited higher rates 
of student on-task participation in group and other stu- 
dent-centered activities than did the mathematics com- 
parison classes. 

The classroom observation data complement the interview 
data. The findings suggest that students in classes using 
Textual Power or Mathematics with Meaning are engaged 
by the work and that the skills in the instructional activities 
require more complex thought processes and strategies than 
are typically demanded. These differences appear more 
dramatic for Mathematics with Meaning, but both Textual 
Power and Mathematics with Meaning have positive impacts 
in the classroom. 

Achievement Findings 

To examine the relationship of instructional materials and 
instructional strategies associated with Textual Power and 
Mathematics with Meaning with student achievement, student 
achievement data at two points in time, spring 2003 and spring 
2004, were analyzed for the same cohort of students. Student 
achievement was measured by state assessments. The same two 
districts — District A and District B — that were analyzed in the 
Year 1 evaluation were analyzed in Year 2. A “pretest/posttest 
with matched control group” research design was utilized to 
determine achievement differentials between Textual Power 
classes and control classes, and Mathematics with Meaning 
classes and control classes. The control classes were matched on 
prior achievement, grade level, and courses. 

Analyses were performed separately for the two school 
districts (Districts A and B). The analyses were conducted 
separately by subject (English and mathematics) and by 
school level (middle and high school). Data were not avail- 
able for District B middle schools; therefore six analyses 
were undertaken. Elierarchical Linear Modeling (E1LM) was 
used for the analyses. The results of the analyses found both 
positive and significant effects on student achievement for 
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2003-04 (Year 2) of the evaluation of Textual Power and 
Mathematics with Meaning at the following levels (see Tables 
11 and 12): 

• District A, High School, Mathematics, p < .05 

• District A, Middle School, Mathematics, p < .05 

• District A, Middle School, English, p < .01 

As the analyses reveal, Mathematics with Meaning and 
Textual Power were associated with higher achievement in 
District A at the middle school in both subjects and at the 
high school in mathematics only. Mathematics with Meaning 
and Textual Power were not associated with higher achieve- 
ment in District B at the high school. 

Year 2 achievement findings reveal significant achieve- 
ment gains at the middle school which were not found in the 
Year 1 achievement analyses. A possible explanation is that 
Textual Power and Mathematics with Meaning were piloted 
in middle schools a year later than in high schools, and this 
suggests that there is a cumulative effect (i.e., more than one 
year) on teaching and learning associated with Textual Power 
and Mathematics with Meaning. 

Utilization of Findings 

The findings associated with the evaluations of Textual 
Power and Mathematics with Meaning in Years 1 and 2 of 
the pilot informed the development (i.e., additional instruc- 


tional materials, enriched instructional strategies materials, 
and enhanced professional development) of their current 
forms in the SpringBoard’" Program. While informing the 
development process, the findings apply to the implementa- 
tion of the instructional materials, strategies, and professional 
development at these points in time (i.e., 2002-03, 2003-04). 
Additional research regarding these components in the larger 
multicomponent SpringBoard Program should be under- 
taken to ascertain their contributions to implementation and 
achievement in SpringBoard classrooms. 


The author is Mary-Margaret Kerns, director of college place- 
ment and retention research. 


References 

Adelman, C. (1999). Answers in the tool box: Academic 
intensity, attendance patterns, and bachelor’s degree attain- 
ment. Washington, D.C.: U.S. Department of Education. 

American Institutes for Research. (2003). Evaluation of 
Mathematics with Meaning and Textual Power pilot pro- 
grams 2002-2003 school year. Washington, D.C.: Author. 


CCollegeBoard 

1 ' connect to college success 


TM 


Table 1 


Table 3 


Sample and Response Rates, Introductory Survey 


In Scope Sample 

English 

108 

Math 

118 

Total 

226 

Respondents by Subject 

English 

83 

77% 

Math 

92 

78% 

Total 

175 

77% 


Sample and Response Rates, Final Survey 


In Scope Sample, Final Survey 

English 

61 

Math 

63 

Site Visit Teachers 

2 

Total 

126 

Respondents by Subject 

English 

53 

87% 

Math 

51 

81% 

Total 

104 

83% 


Table 2 


Number of Reflections Surveys Completed by Teachers 


Number of Surveys 

Number of Teachers 

i 

59 

2 

5 

3 

3 

4 

3 

5 

4 

6 

1 


Table 4 


Regional and School Characteristics of Site Visit Schools 


District 

Region 

Locale 

Middle 
Schools/ 
High Schools 

Teachers 

(MS/ 

HS) 

Program(s) 

D 

mid-Atlantic 

urban 

2 / 2 

5/6 

TP/MWM 

C 

mid-Atlantic 

rural 

1/0 

9 / 0 

TP/MWM 

B 

Western 

urban 

0/3 

0 / 8 

TP/MWM 

F 

Southern 

midsized 

city 

0/1 

0/4 

TP/MWM 

E 

Western 

urban 

0/2 

0 / 6 

TP 

G 

mid-Atlantic 

rural 

2 / 0 

3 / 0 

MWM 


Table 5 

Percentages of Types of Students in Classes Observed 


# of Class 

Mean 
Length of 

% of Obser- 
vations with 

% of Obser- 
vations with 

Mean # of 

Mean # 

Mean # 

Mean # of 

Mean # 

Mean # 

Obser- 

Class in 

Honors 

Remedial 

Students in 

of Asian 

of Black 

Hispanic 

of White 

of Female 

vations 

Minutes 

Students 

Students 

Class 

Students 

Students 

Students 

Students 

Students 


English 


Textual Power 

23 

78 

4% 

0% 

19 

12% 

37% 

27% 

24% 

47% 

English 

Comparison 

13 

75 

15% 

0% 

19 

13% 

38% 

20% 

29% 

50% 


Math 


Mathematics 
with Meaning 

18 

73 

44% 

22% 

21 

7% 

33% 

20% 

39% 

47% 

Math 

Comparison 

12 

79 

33% 

33% 

20 

9% 

20% 

27% 

44% 

53% 


Total 


Total TP and 
MwM Classes 

41 

76 

22% 

10% 

19 

10% 

35% 

24% 

31% 

47% 

Total 

Comparison 

25 

77 

24% 

16% 

20 

11% 

29% 

24% 

36% 

52% 


Total 

66 

76 

23% 

12% 

20 

10% 

33% 

24% 

33% 

49% 
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Table 6 


Collected Sample Distribution 


District 

Region 

Locale 

Math 

ELA 

Pieces of 
Student Work 

Number of Teachers 

Pieces of 
Student Work 

Number of Teachers 

D 

mid-Atlantic 

urban 

27 

3 

47 

4 

C 

mid- Atlantic 

rural 

53 

4 

27 

3 

B 

Western 

urban 

49 

4 

10 

1 

F 

Southern 

midsized city 

10 

1 

19 

2 

E 

Western 

urban 

— 

— 

58 

5 

G 

mid- Atlantic 

rural 

31 

3 

— 

— 


Table 7 


HLM Coefficient Estimates for District A High School 


Math 

Coefficient 

Estimates 

English 

Coefficient 

Estimates 

2002 DSS math achievement score 

0.60*** 

2002 DSS reading achievement score 

0.65*** 

Gender (l=female; 0=male) 

-17.17** 

Gender (l=female; 0=male) 

8.11 

Race (l=white; 0=otherwise) 

2.95 

Race (l=white; 0=otherwise) 

10.94 

Free or reduced lunch 

-19.07* 

Free or reduced lunch 

-25.18 

(l=student has free or reduced lunch; 0=otherwise) 

(l=student has free or reduced lunch; 0=otherwise) 

LEP (l=student is LEP; 0=otherwise) 

-1.62 

LEP (l=student is LEP; 0=otherwise) 

-9.11 

Disability 

(0=no disability or gifted; l=diagnosed with disability) 

-7.31 

Disability 

(0=no disability or gifted; l=diagnosed with disability) 

-8.51 

Teacher Certification 

(l=noncertified subject/talent expert; 0=certified) 

12.92 

Teacher Certification 

(l=noncertified subject/ talent expert; 0=certified) 

-71.66 

Mixed Grade 3 (l=c9; 0=cl0) 

-49.77*** 

Mixed Grade (there is only one group) 

N.A. 

Course (l=Algebra I; 0= Algebra IB) 

24.63* 

Course (there is only one course) 

N.A. 

Treatment group (1 treatment; 0=control) b 

15.18* 

Treatment group (1 treatment; 0=control) c 

45.62** 


Note: *p <.05; **p < .01; ***p < .0001 

“The definition of mixed-grade variable is related to the concentration of different students in different grades in the class. 
b When Math gain scores (2003 scores 2002 scores) were used, p=0.13 for the treatment group coefficient estimate. 

“When English gain scores (2003 scores-2002 scores) were used, similar results were obtained for the treatment group coefficient estimate. 


Table 8 


Regional and School Characteristics of Site Visit Schools 


District 

Region 

Locale 

School level 

Number of schools 
visited 

Number of teach- 
ers observed and 
interviewed 

MWM/TP teachers 
interviewed only 

Program used 

A 

South 

rural 

High 

2 

n 

2 

TP/MWM 

B 

West 

urban 

High 

2 

6 

0 

TP 

F 

South 

midsized city 

Middle 

2 

11 

1 

TP/MWM 

I 

Northeast 

midsized city 

Middle 

2 

6 

0 

MWM 

J 

Southwest 

rural 

High 

1 

3 

1 

TP 

K 

Midwest 

midsized city 

High 

2 

6 

1 

MWM 
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Table 9 


Numbers of English Observations with Descriptive Information, 2003 and 2004 



Number of Class 
Observations 

Mean Length of 
Classes in Minutes 

Mean Length of 
Observations in 
Minutes 

Percent of 
Observations with 
Honors Students 

Percent of 
Observations with 
Remedial Students 

Mean Number of 
Students in Classes 

2003 

Textual Power 

24 

73 

62 

7% 

0% 

19 

English 

Comparison 

12 

66 

64 

13% 

0% 

19 

2004 

Textual Power 

13 

81 

69 

38% 

8% 

26 

English 

Comparison 

7 

80 

70 

29% 

43% 

26 

2003 and 2004 

AU Textual Power 
Classes 

37 

76 

65 

16% 

3% 

21 

AU English 
Comparisons 

19 

71 

66 

21% 

16% 

22 

All English Classes 

56 

74 

65 

18% 

7% 

21 

Table 10 

Numbers of Mathematics Observations with Descriptive Information, 2003 and 2004 


Number of Class 
Observations 

Mean Length of 
Classes in Minutes 

Mean Length of 
Observations in 
Minutes 

Percent of 
Observations with 
Honors Students 

Percent of 
Observations with 
Remedial Students 

Mean Number of 
Students in Classes 

2003 

Mathematics with 
Meaning 

18 

73 

66 

44% 

22% 

21 

Mathematics 

Comparison 

12 

79 

60 

33% 

33% 

20 

2004 

Mathematics with 
Meaning 

15 

78 

64 

27% 

13% 

22 

Mathematics 

Comparison 

8 

79 

64 

25% 

25% 

22 

2003 and 2004 

All MWM Classes 

33 

76 

65 

36% 

18% 

21 

AU Mathematics 
Comparisons 

20 

79 

62 

30% 

30% 

21 

AU Mathematics 
Classes 

53 

77 

64 

34% 

23% 

21 
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Table 11 


HLM Coefficient Estimates for District A High School 


Mathematics 

Coefficient 

Estimate 

English 

Coefficient 

Estimate 

2003 DSS mathematics achievement score (prior achievement) 

0.67*** 

2003 DSS reading achievement score (prior achievement) 

0.92*** 

Gender (l=female; 0=male) 

-16.41** 

Gender (l=female; 0=male) 

26.71 

White (l=white; 0=otherwise) 

1.71 

White (l=white; 0=otherwise) 

72.23 

Hispanic (l=Hispanic; 0=otherwise) 

-16.54 

Hispanic (l=Hispanic; 0=otherwise) 

83.94 

Black (l=black; 0=otherwise) 

-1.57 

Black (l=black; 0=otherwise) 

34.02 

LEP (l=never been considered for LEP; 0=otherwise) 

19.04 

LEP (l=never been considered for LEP; 0=otherwise) 

54.16 

IEP (l=no disability or gifted; 0=diagnosed with disability) 

-3.80 

IEP (l=no disability or gifted; 0=diagnosed with disability) 

59.09 

Grade 10 (l=grade 10; 0=otherwise) 

30.17** 

Grade 10 (l=grade 10; 0=otherwise) 

136.50** 

Grade 11 (l=grade 11; 0=otherwise) 

0.00 

Grade 11 (l=grade 11; 0=otherwise) 

0.00 

Grade 12 (l=grade 12; 0=otherwise) 

0.00 

Course (there is only one course “Eng I”) 

N.A. 

Course 1 (1= Algebra I and class grade 3 C9; 0=otherwise) 

-33.97 

Treatment Group (1 treatment; 0=control) d 

-19.92 

Course 2 (l=Algebra I Hon and class grade C9; 0=otherwise) 

-13.97 


Course 3 (l=Algebra LA and class grade C9; 0=otherwise) 

-64.56** 

Course 4 (1= Algebra LA and class grade P9; 0=otherwise) 

-81.43*** 

Course 5 (1= Algebra IB and class grade C9; 0=otherwise) 

-46.33* 

Course 6 (1= Algebra IB and class grade P9; 0=otherwise) 

-60.21** 

Course 7 (1= Algebra IB and class grade CIO; 0=otherwise) 

-50.01** 

Course 8 (l=Algebra II and class grade CIO; 0=otherwise) 

-17.70 

Course 9 (1= Algebra II and class grade P10; 0=otherwise) 

-13.76 

Treatment Group (1 treatment; 0=control) b 

17.02*' 


Note: *p <.05; **p < .01; ***p < .0001 

a Class Grade is related to the concentration of different students from different grades in a class. 

b When mathematics gain scores (spring 2004 scores-spring 2003 scores) were used, similar results were obtained for the Treatment Group coefficient estimate. 
c For those mathematics coefficient estimates that were significant, their interaction with the Treatment Group variable was tested but the interaction with 
the Treatment Group variable was found to be nonsignificant. 

d When English gain scores (spring 2004 scores-spring 2003 scores) were used, similar results were obtained for the Treatment Group coefficient estimate. 


Table 12 


HLM Coefficient Estimates for District A Middle School 


Mathematics 

Coefficient 

Estimate 

English 

Coefficient 

Estimate 

2003 DSS mathematics achievement score (prior achievement) 

0.47*** 

2003 DSS reading achievement score (prior achievement) 

0.69*** 

Gender (l=female; 0=male) 

-4.33 

Gender (l=female; 0=male) 

-1.99 

White (l=white; 0=otherwise) 

-9.61 

White (l=white; 0=otherwise) 

-6.26 

Hispanic (l=Hispanic; 0=otherwise) 

-18.87 

Hispanic (l=Hispanic; 0=otherwise) 

-20.59 

Black (l=black; 0=otherwise) 

-45.31** 

Black (l=black; 0=otherwise) 

-34.58 

LEP (l=never been considered for LEP; 0=otherwise) 

13.32 

LEP (l=never been considered for LEP; 0=otherwise) 

16.47 

IEP (l=no disability or gifted; 0=diagnosed with disability) 

16.10 

IEP (l=no disability or gifted; 0=diagnosed with disability) 

30.28* 

Grade 7 (l=grade 7; 0=otherwise) 

46.80 

Grade 7 (l=grade 7; 0=otherwise) 

35.40*** 

Grade 8 (l=grade 8; 0=otherwise) 

41.14** 

Grade 8 (l=grade 8; 0=otherwise) 

33.88** 

Course 1 (l=Algebra I Hon; 0=otherwise) 

73.79*** 

Course 1 (1=M/J Lang Arts 1; 0=otherwise) 

0.00 

Course 2 (l=Algebra LA; 0=otherwise) 

44.07*** 

Course 2(1= M/J Lang Arts 2; 0=otherwise) 

0.00 

Course 3 (1= Algebra II Hon; 0=otherwise) 

117.69*** 

Course 3 (1= M/J Lang Arts 3; 0=otherwise) 

16.34 

Course 4 (1=M/J Mathematics 1; 0=otherwise) 

0.00 

Treatment Group b (1 treatment; 0=control) c 

20.88** d 

Course 5 (1=M/J Mathematics 2, Adv; 0=otherwise) 

43.48 


Treatment Group 3 (1 treatment; 0=control) a 

22.72*b 


Note: *p <.05; **p < .01; ***p < .0001 

a When mathematics gain scores (spring 2004 scores-spring 2003 scores) were used, similar results were obtained for the Treatment Group coefficient 
estimate. 

b For those mathematics coefficient estimates that were significant, their interaction with the Treatment Group variable was tested but the interaction with 
the Treatment Group variable was found to be nonsignificant. 

c When English gain scores (spring 2004 scores-spring 2003 scores) were used, similar results were obtained for the Treatment Group coefficient estimate. 
d For those English coefficient estimates that were significant, their interaction with the Treatment Group variable was tested, but the interaction with the 
Treatment Group variable was found to be nonsignificant. 
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